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REMARKS 

Claims 1-3 and 6-25 remain in this application. Claims 4-5 have been canceled. 
Claims 1, 8, and 19 have been amended. By these amendments, no new matter has 
been added. 

Prior to addressing the pending rejections of the claims, it may be helpful to 
briefly review certain innovative aspects of the invention. The invention concerns a 
computer-implemented method for photogrammetry or other reverse rendering 
problems, that has been implemented at a production level as a software tool. As such, 
the invention has proven to have significant advantages over prior art methods and 
tools for photogrammetry in a production environment, by providing substantially faster 
and more accurate solutions to photogrammetry problems. 

One novel aspect of the invention involves the implementation of a solution 
algorithm that computes the full exact Hessian of an error function using automatic 
differentiation and generic programming, applying Newton's method to find the error 
minima in bundle adjustment. Previous approaches, e.g., Levenberg-Marquardt 
iterations or the the "sparse solver" approach described by Dellaert at pages 204-205, 
avoided an exact calculation of the Hessian. The present invention avoids the need to 
employ such approaches, which, whatever their theoretical merits, are difficult to 
program. Using automatic differentiation in a generic environment, programmers who 
are not experts in matrix manipulation and differential calculus can successfully program 
efficient solutions to bundle adjustment problems. 

Another novel aspect of the invention concerns the use of scene graphs, which 
are a type of directed acyclic graph, to define an initial solution estimate. The scene 
graph organizes features of a scene in a hierarchical relationship. This is advantageous 
in manipulating objects in a scene generally. In addition, defining the initial solution for 
a photogrammetry bundle adjustment algorithm using scene graphs provides surprising 
benefits. These benefits include a dramatic reduction in the number of input images 



LA2:759913.1 



Serial No. 10/715,869 
June 23, 2005 
Page 9 

needed to find a solution. By defining the initial solution estimate using a scene graph, 
it is possible to arrive at a solution using a much smaller number of input images. This 
capability may considerably ease the burden of gathering and inputting photogrammetry 
data for video production. 

The invention thereby provides a robust and efficient computer-implemented 
method for solving photogrammetry bundle adjustment problems. Two unconventional 
approaches - employing the Newtonian solution method using the full Hessian 
calculated using automatic differentiation, and use of directed acyclic scene graphs as 
solution estimates - are combined and generate unexpected synergies to provide 
superior results. 

The Examiner rejected Claims 1-25 under 35 U.S.C. § 102(b) in view of Dellaert. 
These rejections are respectfully traversed. 

Dellaert discloses a Monte Carlo expectation-maximization algorithm, and its 
application to photogrammetry and machine vision problems. The Monte Carlo 
approach disclosed by Dellaert is unrelated to the present invention. Delleart also 
discusses various aspects of solving error-minimization problems, but the present 
invention employs different solution techniques than those disclosed in Delleart. Certain 
particular deficiencies of Dellaert are discussed below. 

With respect to Claims 1 and 19, Delleart fails to disclose or suggest any use of 
directed acyclic scene graphs ("DAG's"). Delleart therefore does not and cannot 
disclose: 

receiving a first directed acyclic graph of the scene, the first 
directed acyclic graph defining transforms between hierarchically-related 
nodes of the graph; 

receiving user input indicating a plurality of corresponding features 
each appearing in at least two of the plurality of photographic images and 
associated with a node of the first directed acylic graph; 
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determining an error function for a reverse-rendering function, the 
reverse-rendering function defining a relationship consistent with the first 
directed acylic graph between three-dimensional coordinates in the three- 
dimensional scene and corresponding two-dimensional coordinates of the 
plurality of corresponding features; 
as defined by Claims 1 and 19. 

In paragraph 2e, the Examiner equated use of a JPDAF as disclosed at page 44 
of Dellaert to receiving a scene graph comprising a transform between a parent object 
and a child object, as defined by former Claim 5. The Joint Probability Data Association 
Filter method or "JPDAF", is used in motion tracking applications to track multiple 
moving objects from 2-D vision data. JPDAF updates a target state for multiple moving 
targets weighted with associated posterior hypotheses from measurements in a current 
observation. JPDAF is a distinctly different technique from use of directed acyclic 
graphs to model a scene. Claims 1 and 19 define use of a scene graph or DAG to 
provide an initial solution estimate and guide the error minimization process. Unlike a 
DAG, a JPDAF does not provide a solution estimate with defined, hierarchical 
relationships between objects in a solution estimate. Such an approach would have no 
utility for real-time target-tracking applications for which JPDAF is designed. Instead, a 
JPDAF filters possible multiple-target solutions to 2-D vision estimates using probability 
estimates. Dellaert therefore fails to disclose or to suggest the use of DAG's as defined 
by Claims 1 and 19, and these rejections should be withdrawn. Likewise, Claims 2-3, 6- 
12 and 20-25 should also be allowable, at least as depending from an allowable base 
claim. Claims 14 and 15 define similar limitations, and should also be allowable. 

Further with respect to Claims 6, 18 and 23, Delleart teaches away from 
calculation of an exact Hessian, or guiding a solution using Newton's method. Instead, 
Delleart proposes using a sparse solver technique or Levenberg-Marquardt approach 
for error minimization. See pages 203-205. Delleart and others of ordinary skill would 
have regarded computing the full Hessian and employing Newton's method at best as 
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an inefficient, slow solution technique. For this reason, the prior art consistently teaches 
avoiding computation of the full Hessian. The present invention contradicts 
conventional wisdom to provide an efficient solution using generic programming and 
automatic differentiation. Claims 6, 18 and 23 should therefore be allowable, for this 
additional reason. 

With respect to Claims 3, 13, and 25, Delleart fails to disclose or suggest placing 
a camera supplying photogrammetry data within a three dimensional scene to be 
modeled. Applicant believes that such steps were avoided in prior-art photogrammetry, 
because data from cameras within the scene made solutions too difficult using prior-art 
solution techniques. Whether or not the general mathematical approach taught by 
Delleart or others is, in hindsight, theoretically capable of handing data from cameras 
placed within a scene is not relevant. The relevant fact is that cameras were 
traditionally always placed outside of the scene to be modeled. Indeed, Delleart 
discloses no example or suggestion of using photographic data collected inside of a 
scene for photogrammetry. Claims 3, 13, and 25 should therefore be allowable for this 
additional reason. Claims 14-17 should further be allowable, at least as depending from 
allowable Claim 13. 

In view of the foregoing, the Applicant respectfully submits that Claims 1-3 and 6- 
25 are in condition for allowance. Reconsideration and withdrawal of the rejections is 
respectfully requested, and a timely Notice of Allowability is solicited. If it would be 
helpful to placing this application in condition for allowance, the Applicant encourages 
the Examiner to contact the undersigned counsel and conduct a telephonic interview. 
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To the extent necessary, Applicant petitions the Commissioner for a one-month 
extension of time, extending to July 15, 2005, the period for response to the Office 
Action dated March 15, 2005. A check in the amount of $120.00 is enclosed for the 
one-month extension of time pursuant to 37 CFR §1.1 7(a)(1). The Commissioner is 
authorized to charge any shortage in fees due in connection with the filing of this paper, 
including extension of time fees, to Deposit Account No. 50-0639. 



O'MELVENY & MYERS LLP 

• 400 South Hope Street 

Los Angeles, CA 90071-2899 
Telephone: (213) 430-6000 
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Respectfully submitted, 





Brian M. Berliner 
Attorney for Applicant 
Registration No. 34,549 



